2025-10-15 19:56:14,308 [ 577526 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2025-10-15 19:56:14,308 [ 577526 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2025-10-15 19:56:14,308 [ 577526 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2025-10-15 19:56:14,308 [ 577526 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_vs3hxi --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=1e0b53d756cf -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_18]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_19]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_20]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_21]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_22]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_23]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_24]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_25]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_5]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_7]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_80]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_81]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_9]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_default]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default]' test_mysql57_database_engine/test.py::test_password_leak test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_mysql57_database_engine/test.py::test_restart_server test_named_collections/test.py::test_config_reload test_named_collections/test.py::test_default_access test_named_collections/test.py::test_granular_access_create_alter_drop_query test_named_collections/test.py::test_granular_access_show_query test_named_collections/test.py::test_keeper_storage 'test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1]' 'test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0]' 'test_named_collections/test.py::test_name_escaping[node]' 'test_named_collections/test.py::test_name_escaping[node_with_keeper]' test_named_collections/test.py::test_show_grants 'test_named_collections/test.py::test_sql_commands[False]' 'test_named_collections/test.py::test_sql_commands[True]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1]' test_part_uuid/test.py::test_part_uuid test_partition/test.py::test_attach_check_all_parts test_partition/test.py::test_cannot_attach_active_part test_partition/test.py::test_detached_part_dir_exists test_partition/test.py::test_drop_detached_parts test_partition/test.py::test_make_clone_in_detached test_partition/test.py::test_partition_complex test_partition/test.py::test_partition_simple test_partition/test.py::test_system_detached_parts test_placement_info/test.py::test_placement_info_from_config test_placement_info/test.py::test_placement_info_from_file test_placement_info/test.py::test_placement_info_from_imds test_placement_info/test.py::test_placement_info_missing_data test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica test_recovery_replica/test.py::test_choose_source_replica test_recovery_replica/test.py::test_recovery test_recovery_replica/test.py::test_update_metadata test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop test_reload_query_masking_rules/test.py::test_reload_query_masking_rules test_reload_zookeeper/test.py::test_reload_zookeeper test_reloading_settings_from_users_xml/test.py::test_force_reload test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_merge_tree_encryption_codec/test.py::test_different_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces test_replicated_merge_tree_encryption_codec/test.py::test_same_keys -vvv" altinityinfra/integration-tests-runner:37a9815fd2fa '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: order-1.0.1, random-0.2, timeout-2.2.0, repeat-0.9.3, reportlog-0.4.0, xdist-3.5.0 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0] test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum test_partition/test.py::test_attach_check_all_parts test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_named_collections/test.py::test_config_reload test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] test_recovery_replica/test.py::test_choose_source_replica test_reloading_settings_from_users_xml/test.py::test_force_reload test_placement_info/test.py::test_placement_info_from_config test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table [gw6] [ 1%] PASSED test_reloading_settings_from_users_xml/test.py::test_force_reload test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout [gw7] [ 2%] PASSED test_placement_info/test.py::test_placement_info_from_config test_placement_info/test.py::test_placement_info_from_file [gw3] [ 3%] PASSED test_partition/test.py::test_attach_check_all_parts test_partition/test.py::test_cannot_attach_active_part [gw6] [ 4%] PASSED test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum [gw3] [ 5%] PASSED test_partition/test.py::test_cannot_attach_active_part test_partition/test.py::test_detached_part_dir_exists [gw7] [ 6%] PASSED test_placement_info/test.py::test_placement_info_from_file test_placement_info/test.py::test_placement_info_from_imds [gw6] [ 7%] PASSED test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int [gw6] [ 8%] PASSED test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload [gw3] [ 9%] PASSED test_partition/test.py::test_detached_part_dir_exists test_partition/test.py::test_drop_detached_parts [gw7] [ 10%] PASSED test_placement_info/test.py::test_placement_info_from_imds test_placement_info/test.py::test_placement_info_missing_data [gw7] [ 11%] PASSED test_placement_info/test.py::test_placement_info_missing_data [gw9] [ 12%] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_merge_tree_encryption_codec/test.py::test_different_keys [gw6] [ 13%] PASSED test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout [gw6] [ 14%] PASSED test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout [gw9] [ 15%] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces [gw0] [ 16%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] [gw3] [ 17%] PASSED test_partition/test.py::test_drop_detached_parts test_partition/test.py::test_make_clone_in_detached test_part_uuid/test.py::test_part_uuid [gw0] [ 18%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop [gw9] [ 19%] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces test_replicated_merge_tree_encryption_codec/test.py::test_same_keys [gw0] [ 20%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] [gw3] [ 21%] PASSED test_partition/test.py::test_make_clone_in_detached test_partition/test.py::test_partition_complex [gw2] [ 22%] PASSED test_named_collections/test.py::test_config_reload test_named_collections/test.py::test_default_access [gw0] [ 23%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] [gw3] [ 24%] PASSED test_partition/test.py::test_partition_complex test_partition/test.py::test_partition_simple [gw9] [ 25%] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_same_keys [gw0] [ 26%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] [gw3] [ 27%] PASSED test_partition/test.py::test_partition_simple test_partition/test.py::test_system_detached_parts [gw8] [ 28%] PASSED test_recovery_replica/test.py::test_choose_source_replica test_recovery_replica/test.py::test_recovery [gw0] [ 29%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] [gw1] [ 30%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1] [gw0] [ 31%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] [gw4] [ 32%] PASSED test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] [gw0] [ 33%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] [gw8] [ 34%] PASSED test_recovery_replica/test.py::test_recovery test_recovery_replica/test.py::test_update_metadata [gw5] [ 35%] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server [gw1] [ 36%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0] [gw0] [ 37%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] [gw4] [ 38%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper [gw6] [ 39%] PASSED test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop [gw0] [ 40%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] [gw4] [ 41%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection [gw0] [ 42%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] [gw1] [ 43%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1] [gw3] [ 44%] PASSED test_partition/test.py::test_system_detached_parts test_reload_zookeeper/test.py::test_reload_zookeeper [gw0] [ 45%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] [gw7] [ 46%] PASSED test_part_uuid/test.py::test_part_uuid [gw0] [ 47%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] [gw2] [ 48%] PASSED test_named_collections/test.py::test_default_access test_named_collections/test.py::test_granular_access_create_alter_drop_query [gw0] [ 49%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] [gw1] [ 50%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0] [gw0] [ 51%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] [gw2] [ 52%] PASSED test_named_collections/test.py::test_granular_access_create_alter_drop_query test_named_collections/test.py::test_granular_access_show_query test_reload_query_masking_rules/test.py::test_reload_query_masking_rules [gw0] [ 53%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] [gw0] [ 54%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] [gw1] [ 55%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1] [gw5] [ 56%] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table [gw0] [ 57%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] [gw0] [ 58%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] [gw7] [ 59%] PASSED test_reload_query_masking_rules/test.py::test_reload_query_masking_rules [gw8] [ 60%] PASSED test_recovery_replica/test.py::test_update_metadata [gw0] [ 61%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] [gw1] [ 62%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0] [gw0] [ 63%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings [gw0] [ 64%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] [gw5] [ 65%] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server [gw1] [ 66%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0] test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1] [gw0] [ 67%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] [gw6] [ 68%] PASSED test_reload_zookeeper/test.py::test_reload_zookeeper [gw0] [ 69%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] [gw0] [ 70%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] [gw1] [ 71%] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1] [gw2] [ 72%] PASSED test_named_collections/test.py::test_granular_access_show_query test_named_collections/test.py::test_keeper_storage [gw0] [ 73%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] [gw0] [ 74%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] [gw0] [ 75%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] [gw0] [ 76%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] test_mysql57_database_engine/test.py::test_mysql_types[float_1] [gw7] [ 77%] PASSED test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings [gw0] [ 78%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_1] test_mysql57_database_engine/test.py::test_mysql_types[float_2] [gw0] [ 79%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_2] test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] [gw0] [ 80%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] [gw5] [ 81%] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch [gw0] [ 82%] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] test_mysql57_database_engine/test.py::test_password_leak [gw0] [ 83%] PASSED test_mysql57_database_engine/test.py::test_password_leak test_mysql57_database_engine/test.py::test_predefined_connection_configuration [gw9] [ 84%] PASSED test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper [gw0] [ 85%] PASSED test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_mysql57_database_engine/test.py::test_restart_server [gw0] [ 86%] PASSED test_mysql57_database_engine/test.py::test_restart_server [gw5] [ 87%] PASSED test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch [gw2] [ 88%] PASSED test_named_collections/test.py::test_keeper_storage test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1] [gw2] [ 89%] PASSED test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1] test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0] [gw2] [ 90%] PASSED test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0] test_named_collections/test.py::test_name_escaping[node] [gw2] [ 91%] PASSED test_named_collections/test.py::test_name_escaping[node] test_named_collections/test.py::test_name_escaping[node_with_keeper] [gw2] [ 92%] PASSED test_named_collections/test.py::test_name_escaping[node_with_keeper] test_named_collections/test.py::test_show_grants [gw2] [ 93%] PASSED test_named_collections/test.py::test_show_grants test_named_collections/test.py::test_sql_commands[False] [gw2] [ 94%] PASSED test_named_collections/test.py::test_sql_commands[False] test_named_collections/test.py::test_sql_commands[True] [gw2] [ 95%] PASSED test_named_collections/test.py::test_sql_commands[True] [gw4] [ 96%] FAILED test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] [gw4] [ 97%] FAILED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] [gw4] [ 98%] FAILED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] test_quorum_inserts/test.py::test_insert_quorum_with_ttl [gw4] [ 99%] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica [gw4] [100%] PASSED test_quorum_inserts/test.py::test_simple_add_replica =================================== FAILURES =================================== ________________ test_insert_quorum_with_keeper_loss_connection ________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 def test_insert_quorum_with_keeper_loss_connection(): zero.query( "DROP TABLE IF EXISTS test_insert_quorum_with_keeper_fail ON CLUSTER cluster" ) create_query = ( "CREATE TABLE test_insert_quorum_with_keeper_loss" "(a Int8, d Date) " "Engine = ReplicatedMergeTree('/clickhouse/tables/{table}', '{replica}') " "ORDER BY a " ) zero.query(create_query) first.query(create_query) first.query("SYSTEM STOP FETCHES test_insert_quorum_with_keeper_loss") zero.query("SYSTEM ENABLE FAILPOINT replicated_merge_tree_commit_zk_fail_after_op") zero.query("SYSTEM ENABLE FAILPOINT replicated_merge_tree_insert_retry_pause") with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor: insert_future = executor.submit( lambda: zero.query( "INSERT INTO test_insert_quorum_with_keeper_loss(a,d) VALUES(1, '2011-01-01')", settings={"insert_quorum_timeout": 150000}, ) ) pm = PartitionManager() pm.drop_instance_zk_connections(zero) retries = 0 zk = cluster.get_kazoo_client("zoo1") while True: if ( zk.exists( "/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active" ) is None ): break print("replica is still active") time.sleep(1) retries += 1 if retries == 120: raise Exception("Can not wait cluster replica inactive") first.query("SYSTEM ENABLE FAILPOINT finish_set_quorum_failed_parts") quorum_fail_future = executor.submit( lambda: first.query( "SYSTEM WAIT FAILPOINT finish_set_quorum_failed_parts", timeout=300 ) ) first.query("SYSTEM START FETCHES test_insert_quorum_with_keeper_loss") concurrent.futures.wait([quorum_fail_future]) > assert quorum_fail_future.exception() is None E AssertionError: assert QueryTimeoutExceedException('Client timed out!') is None E + where QueryTimeoutExceedException('Client timed out!') = >() E + where > = .exception test_quorum_inserts/test.py:424: AssertionError ----------------------------- Captured stdout call ----------------------------- replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active replica is still active ------------------------------ Captured log call ------------------------------- 2025-10-15 19:57:09 [ 526 ] DEBUG : Executing query DROP TABLE IF EXISTS test_insert_quorum_with_keeper_fail ON CLUSTER cluster on zero (cluster.py:3602, query) 2025-10-15 19:57:09 [ 526 ] DEBUG : Executing query CREATE TABLE test_insert_quorum_with_keeper_loss(a Int8, d Date) Engine = ReplicatedMergeTree('/clickhouse/tables/{table}', '{replica}') ORDER BY a on zero (cluster.py:3602, query) 2025-10-15 19:57:09 [ 526 ] DEBUG : Executing query CREATE TABLE test_insert_quorum_with_keeper_loss(a Int8, d Date) Engine = ReplicatedMergeTree('/clickhouse/tables/{table}', '{replica}') ORDER BY a on first (cluster.py:3602, query) 2025-10-15 19:57:10 [ 526 ] DEBUG : Executing query SYSTEM STOP FETCHES test_insert_quorum_with_keeper_loss on first (cluster.py:3602, query) 2025-10-15 19:57:10 [ 526 ] DEBUG : Executing query SYSTEM ENABLE FAILPOINT replicated_merge_tree_commit_zk_fail_after_op on zero (cluster.py:3602, query) 2025-10-15 19:57:10 [ 526 ] DEBUG : Executing query SYSTEM ENABLE FAILPOINT replicated_merge_tree_insert_retry_pause on zero (cluster.py:3602, query) 2025-10-15 19:57:10 [ 526 ] DEBUG : Executing query INSERT INTO test_insert_quorum_with_keeper_loss(a,d) VALUES(1, '2011-01-01') on zero (cluster.py:3602, query) 2025-10-15 19:57:10 [ 526 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2025-10-15 19:57:10 [ 526 ] DEBUG : No config file found (config.py:28, find_config_file) 2025-10-15 19:57:10 [ 526 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2025-10-15 19:57:10 [ 526 ] DEBUG : No config file found (config.py:28, find_config_file) 2025-10-15 19:57:10 [ 526 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 824 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "POST /v1.42/containers/create HTTP/1.1" 201 88 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "GET /v1.42/containers/6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "POST /v1.42/containers/6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d/start HTTP/1.1" 204 0 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : [network] Created new container 6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d (network.py:289, _ensure_container) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "POST /v1.42/containers/6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d/exec HTTP/1.1" 201 74 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "POST /v1.42/exec/2e4f188ef9fcab556f05a6f8d54e50c0bb758e1cb985963a3daea70b936e4c03/start HTTP/1.1" 101 0 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "GET /v1.42/exec/2e4f188ef9fcab556f05a6f8d54e50c0bb758e1cb985963a3daea70b936e4c03/json HTTP/1.1" 200 461 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : [network] 6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d: ['iptables', '--wait', '-I', 'DOCKER-USER', '1', '-p', 'tcp', '-s', '172.16.10.6', '--dport', '2181', '-j', 'DROP'] (0): (network.py:308, _exec_run) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "POST /v1.42/containers/6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d/exec HTTP/1.1" 201 74 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "POST /v1.42/exec/29639961a2399324faa16ab71041d494d14e2b5d5a917fe32f079dccf25a0d0f/start HTTP/1.1" 101 0 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "GET /v1.42/exec/29639961a2399324faa16ab71041d494d14e2b5d5a917fe32f079dccf25a0d0f/json HTTP/1.1" 200 461 (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : [network] 6d3d20b9e337f064f35c1abfc683952d2542d5f05014fef5f27d7c7ff8ef051d: ['iptables', '--wait', '-I', 'DOCKER-USER', '1', '-p', 'tcp', '-d', '172.16.10.6', '--sport', '2181', '-j', 'DROP'] (0): (network.py:308, _exec_run) 2025-10-15 19:57:11 [ 526 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2135, get_instance_ip) 2025-10-15 19:57:11 [ 526 ] DEBUG : http://localhost:None "GET /v1.42/containers/roottestquoruminserts_gw4_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2025-10-15 19:57:11 [ 526 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.10.4, port:2181, use_ssl:False (cluster.py:3286, get_kazoo_client) 2025-10-15 19:57:11 [ 526 ] INFO : Connecting to 172.16.10.4(172.16.10.4):2181, use_ssl: False (connection.py:650, _connect) 2025-10-15 19:57:11 [ 526 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2025-10-15 19:57:11 [ 526 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2025-10-15 19:57:11 [ 526 ] DEBUG : Sending request(xid=1): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:11 [ 526 ] DEBUG : Received response(xid=1): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:12 [ 526 ] DEBUG : Sending request(xid=2): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:12 [ 526 ] DEBUG : Received response(xid=2): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:13 [ 526 ] DEBUG : Sending request(xid=3): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:13 [ 526 ] DEBUG : Received response(xid=3): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:14 [ 526 ] DEBUG : Sending request(xid=4): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:14 [ 526 ] DEBUG : Received response(xid=4): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:15 [ 526 ] DEBUG : Sending request(xid=5): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:15 [ 526 ] DEBUG : Received response(xid=5): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:16 [ 526 ] DEBUG : Sending request(xid=6): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:16 [ 526 ] DEBUG : Received response(xid=6): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:17 [ 526 ] DEBUG : Sending request(xid=7): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:17 [ 526 ] DEBUG : Received response(xid=7): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:18 [ 526 ] DEBUG : Sending request(xid=8): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:18 [ 526 ] DEBUG : Received response(xid=8): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:19 [ 526 ] DEBUG : Sending request(xid=9): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:19 [ 526 ] DEBUG : Received response(xid=9): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:20 [ 526 ] DEBUG : Sending request(xid=10): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:20 [ 526 ] DEBUG : Received response(xid=10): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:21 [ 526 ] DEBUG : Sending request(xid=11): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:21 [ 526 ] DEBUG : Received response(xid=11): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:21 [ 526 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2025-10-15 19:57:21 [ 526 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2025-10-15 19:57:21 [ 526 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2025-10-15 19:57:21 [ 526 ] INFO : Connecting to 172.16.10.4(172.16.10.4):2181, use_ssl: False (connection.py:650, _connect) 2025-10-15 19:57:21 [ 526 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=397, time_out=10000, session_id=7, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2025-10-15 19:57:21 [ 526 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2025-10-15 19:57:22 [ 526 ] DEBUG : Sending request(xid=1): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:22 [ 526 ] DEBUG : Received response(xid=1): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:23 [ 526 ] DEBUG : Sending request(xid=2): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:23 [ 526 ] DEBUG : Received response(xid=2): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:24 [ 526 ] DEBUG : Sending request(xid=3): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:24 [ 526 ] DEBUG : Received response(xid=3): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:25 [ 526 ] DEBUG : Sending request(xid=4): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:25 [ 526 ] DEBUG : Received response(xid=4): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:26 [ 526 ] DEBUG : Sending request(xid=5): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:26 [ 526 ] DEBUG : Received response(xid=5): ZnodeStat(czxid=368, mzxid=368, ctime=1760558229825, mtime=1760558229825, version=0, cversion=0, aversion=0, ephemeralOwner=5, dataLength=43, numChildren=0, pzxid=368) (connection.py:410, _read_response) 2025-10-15 19:57:27 [ 526 ] DEBUG : Sending request(xid=6): Exists(path='/clickhouse/tables/test_insert_quorum_with_keeper_loss/replicas/zero/is_active', watcher=None) (connection.py:312, _submit) 2025-10-15 19:57:27 [ 526 ] DEBUG : Executing query SYSTEM ENABLE FAILPOINT finish_set_quorum_failed_parts on first (cluster.py:3602, query) 2025-10-15 19:57:27 [ 526 ] DEBUG : Executing query SYSTEM WAIT FAILPOINT finish_set_quorum_failed_parts on first (cluster.py:3602, query) 2025-10-15 19:57:27 [ 526 ] DEBUG : Executing query SYSTEM START FETCHES test_insert_quorum_with_keeper_loss on first (cluster.py:3602, query) 2025-10-15 20:02:27 [ 526 ] DEBUG : Timed out. Last stdout:, stderr: (client.py:233, get_answer) ________________ test_insert_quorum_with_move_partition[False] _________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = add_new_data = False @pytest.mark.parametrize(("add_new_data"), [False, True]) def test_insert_quorum_with_move_partition(started_cluster, add_new_data): # use different table names for easier disambiguation in logs between runs (you may also check uuid though, but not always convenient) source_table_name = ( "test_insert_quorum_with_move_partition_source_new_data" if add_new_data else "test_insert_quorum_with_move_partition_source" ) destination_table_name = ( "test_insert_quorum_with_move_partition_destination_new_data" if add_new_data else "test_insert_quorum_with_move_partition_destination" ) > zero.query(f"DROP TABLE IF EXISTS {source_table_name} ON CLUSTER cluster") test_quorum_inserts/test.py:226: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 231, stderr: Received exception from server (version 24.8.14): E Code: 999. DB::Exception: Received from 172.16.10.6:9000. Coordination::Exception. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 172.16.10.2:2181, 172.16.10.3:2181, 172.16.10.4:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.2:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.2:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.3:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.3:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.4:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.4:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.2:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.2:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.3:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.3:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.4:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.4:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.2:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.2:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.3:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.3:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.4:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.4:2181 E . Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034362854 E 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb22c9 E 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa90445 E 3. ./src/Common/LoggingFormatStringHelpers.h:45: Coordination::Exception::Exception(Coordination::Error, FormatStringHelperImpl::type>, String&) @ 0x000000002b911c91 E 4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:573: Coordination::ZooKeeper::connect(std::vector> const&, Poco::Timespan) @ 0x000000002e6c5b2d E 5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr) @ 0x000000002e6bef00 E 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs, std::unique_ptr>) @ 0x000000002e5f0bd6 E 7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:232: zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr, std::vector>, std::unique_ptr>) @ 0x000000002e5f673e E 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:456: zkutil::ZooKeeper::startNewSession() const @ 0x000000002e611237 E 9. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getZooKeeper() const @ 0x0000000028553bce E 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DDLWorker::getAndSetZooKeeper() @ 0x000000002867b852 E 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1057: DB::DDLWorker::enqueueQuery(DB::DDLLogEntry&) @ 0x000000002869cc18 E 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::executeDDLQueryOnCluster(std::shared_ptr const&, std::shared_ptr, DB::DDLQueryOnClusterParams const&) @ 0x0000000029b07b1b E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291f2c76 E 14. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291f20ad E 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3cc5c E 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b35fe5 E 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d186c4c E 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1baa80 E 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003453ffef E 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000034540bd7 E 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x00000000344432eb E 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003443d448 E 23. asan_thread_start(void*) @ 0x000000000aa45059 E 24. ? @ 0x00007f56332aaac3 E 25. ? @ 0x00007f563333c850 E . (KEEPER_EXCEPTION) E (query: DROP TABLE IF EXISTS test_insert_quorum_with_move_partition_source ON CLUSTER cluster) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2025-10-15 20:02:28 [ 526 ] DEBUG : Executing query DROP TABLE IF EXISTS test_insert_quorum_with_move_partition_source ON CLUSTER cluster on zero (cluster.py:3602, query) _________________ test_insert_quorum_with_move_partition[True] _________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 started_cluster = add_new_data = True @pytest.mark.parametrize(("add_new_data"), [False, True]) def test_insert_quorum_with_move_partition(started_cluster, add_new_data): # use different table names for easier disambiguation in logs between runs (you may also check uuid though, but not always convenient) source_table_name = ( "test_insert_quorum_with_move_partition_source_new_data" if add_new_data else "test_insert_quorum_with_move_partition_source" ) destination_table_name = ( "test_insert_quorum_with_move_partition_destination_new_data" if add_new_data else "test_insert_quorum_with_move_partition_destination" ) > zero.query(f"DROP TABLE IF EXISTS {source_table_name} ON CLUSTER cluster") test_quorum_inserts/test.py:226: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3603: in query return self.client.query( helpers/client.py:36: in wrap return func(self, *args, **kwargs) helpers/client.py:74: in query ).get_answer() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = def get_answer(self): self.process.wait(timeout=DEFAULT_QUERY_TIMEOUT) self.stdout_file.seek(0) self.stderr_file.seek(0) stdout = self.stdout_file.read().decode("utf-8", errors="replace") stderr = self.stderr_file.read().decode("utf-8", errors="replace") if ( self.timer is not None and not self.process_finished_before_timeout and not self.ignore_error ): logging.debug(f"Timed out. Last stdout:{stdout}, stderr:{stderr}") raise QueryTimeoutExceedException("Client timed out!") if ( self.process.returncode != 0 or self.remove_trash_from_stderr(stderr) ) and not self.ignore_error: > raise QueryRuntimeException( "Client failed! Return code: {}, stderr: {}".format( self.process.returncode, stderr ), self.process.returncode, stderr, ) E helpers.client.QueryRuntimeException: Client failed! Return code: 231, stderr: Received exception from server (version 24.8.14): E Code: 999. DB::Exception: Received from 172.16.10.6:9000. Coordination::Exception. Coordination::Exception: All connection tries failed while connecting to ZooKeeper. nodes: 172.16.10.2:2181, 172.16.10.4:2181, 172.16.10.3:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.2:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.2:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.4:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.4:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.3:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.3:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.2:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.2:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.4:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.4:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.3:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.3:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.2:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.2:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.4:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.4:2181 E Poco::Exception. Code: 1000, e.code() = 0, Timeout: connect timed out: 172.16.10.3:2181 (version 24.8.14.10521.altinitytest (altinity build)), 172.16.10.3:2181 E . Stack trace: E E 0. ./contrib/llvm-project/libcxx/include/exception:141: Poco::Exception::Exception(String const&, int) @ 0x0000000034362854 E 1. ./build_docker/./src/Common/Exception.cpp:111: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000001adb22c9 E 2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000aa90445 E 3. ./src/Common/LoggingFormatStringHelpers.h:45: Coordination::Exception::Exception(Coordination::Error, FormatStringHelperImpl::type>, String&) @ 0x000000002b911c91 E 4. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:573: Coordination::ZooKeeper::connect(std::vector> const&, Poco::Timespan) @ 0x000000002e6c5b2d E 5. ./build_docker/./src/Common/ZooKeeper/ZooKeeperImpl.cpp:0: Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr) @ 0x000000002e6bef00 E 6. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:0: zkutil::ZooKeeper::init(zkutil::ZooKeeperArgs, std::unique_ptr>) @ 0x000000002e5f0bd6 E 7. ./build_docker/./src/Common/ZooKeeper/ZooKeeper.cpp:232: zkutil::ZooKeeper::ZooKeeper(zkutil::ZooKeeperArgs const&, std::shared_ptr, std::vector>, std::unique_ptr>) @ 0x000000002e5f673e E 8. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:456: zkutil::ZooKeeper::startNewSession() const @ 0x000000002e611237 E 9. ./build_docker/./src/Interpreters/Context.cpp:0: DB::Context::getZooKeeper() const @ 0x0000000028553bce E 10. ./contrib/llvm-project/libcxx/include/__utility/swap.h:37: DB::DDLWorker::getAndSetZooKeeper() @ 0x000000002867b852 E 11. ./build_docker/./src/Interpreters/DDLWorker.cpp:1057: DB::DDLWorker::enqueueQuery(DB::DDLLogEntry&) @ 0x000000002869cc18 E 12. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:593: DB::executeDDLQueryOnCluster(std::shared_ptr const&, std::shared_ptr, DB::DDLQueryOnClusterParams const&) @ 0x0000000029b07b1b E 13. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:701: DB::InterpreterDropQuery::executeSingleDropQuery(std::shared_ptr const&) @ 0x00000000291f2c76 E 14. ./build_docker/./src/Interpreters/InterpreterDropQuery.cpp:68: DB::InterpreterDropQuery::execute() @ 0x00000000291f20ad E 15. ./build_docker/./src/Interpreters/executeQuery.cpp:0: DB::executeQueryImpl(char const*, char const*, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000029b3cc5c E 16. ./build_docker/./src/Interpreters/executeQuery.cpp:1397: DB::executeQuery(String const&, std::shared_ptr, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x0000000029b35fe5 E 17. ./contrib/llvm-project/libcxx/include/__memory/shared_ptr.h:612: DB::TCPHandler::runImpl() @ 0x000000002d186c4c E 18. ./build_docker/./src/Server/TCPHandler.cpp:2517: DB::TCPHandler::run() @ 0x000000002d1baa80 E 19. ./build_docker/./base/poco/Net/src/TCPServerConnection.cpp:57: Poco::Net::TCPServerConnection::start() @ 0x000000003453ffef E 20. ./contrib/llvm-project/libcxx/include/__memory/unique_ptr.h:48: Poco::Net::TCPServerDispatcher::run() @ 0x0000000034540bd7 E 21. ./build_docker/./base/poco/Foundation/src/ThreadPool.cpp:219: Poco::PooledThread::run() @ 0x00000000344432eb E 22. ./base/poco/Foundation/include/Poco/SharedPtr.h:231: Poco::ThreadImpl::runnableEntry(void*) @ 0x000000003443d448 E 23. asan_thread_start(void*) @ 0x000000000aa45059 E 24. ? @ 0x00007f56332aaac3 E 25. ? @ 0x00007f563333c850 E . (KEEPER_EXCEPTION) E (query: DROP TABLE IF EXISTS test_insert_quorum_with_move_partition_source_new_data ON CLUSTER cluster) helpers/client.py:239: QueryRuntimeException ------------------------------ Captured log call ------------------------------- 2025-10-15 20:03:03 [ 526 ] DEBUG : Executing query DROP TABLE IF EXISTS test_insert_quorum_with_move_partition_source_new_data ON CLUSTER cluster on zero (cluster.py:3602, query) ============================== slowest durations =============================== 318.61s call test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 72.31s call test_named_collections/test.py::test_keeper_storage 52.59s call test_named_collections/test.py::test_sql_commands[True] 39.89s call test_named_collections/test.py::test_sql_commands[False] 35.92s setup test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 35.87s call test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 35.56s call test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 34.97s setup test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper 32.24s setup test_recovery_replica/test.py::test_choose_source_replica 32.05s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0] 31.93s setup test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table 31.39s setup test_named_collections/test.py::test_config_reload 29.91s call test_recovery_replica/test.py::test_update_metadata 29.19s call test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper 28.70s call test_named_collections/test.py::test_granular_access_show_query 25.21s call test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server 24.86s call test_named_collections/test.py::test_default_access 24.29s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] 24.03s call test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server 22.62s teardown test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper 21.19s call test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch 21.06s setup test_part_uuid/test.py::test_part_uuid 19.85s setup test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings 19.18s call test_reload_zookeeper/test.py::test_reload_zookeeper 18.87s call test_quorum_inserts/test.py::test_insert_quorum_with_ttl 18.40s setup test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids 16.39s setup test_placement_info/test.py::test_placement_info_from_config 15.24s setup test_partition/test.py::test_attach_check_all_parts 15.15s call test_partition/test.py::test_system_detached_parts 14.67s call test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table 14.45s setup test_reloading_settings_from_users_xml/test.py::test_force_reload 13.27s call test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table 12.97s teardown test_quorum_inserts/test.py::test_simple_add_replica 12.73s teardown test_recovery_replica/test.py::test_update_metadata 12.40s teardown test_reload_zookeeper/test.py::test_reload_zookeeper 12.37s setup test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop 11.88s setup test_reload_zookeeper/test.py::test_reload_zookeeper 10.68s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1] 10.16s teardown test_replicated_merge_tree_encryption_codec/test.py::test_same_keys 9.06s teardown test_part_uuid/test.py::test_part_uuid 8.70s call test_mysql57_database_engine/test.py::test_restart_server 8.54s call test_named_collections/test.py::test_name_escaping[node_with_keeper] 8.13s teardown test_named_collections/test.py::test_sql_commands[True] 7.86s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0] 7.73s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1] 7.71s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0] 7.67s call test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 7.67s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0] 7.55s setup test_reload_query_masking_rules/test.py::test_reload_query_masking_rules 7.28s teardown test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch 7.21s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0] 7.09s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1] 6.94s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1] 6.85s call test_recovery_replica/test.py::test_choose_source_replica 6.70s teardown test_partition/test.py::test_system_detached_parts 6.65s call test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1] 6.45s call test_named_collections/test.py::test_show_grants 6.44s teardown test_reload_query_masking_rules/test.py::test_reload_query_masking_rules 6.28s call test_named_collections/test.py::test_name_escaping[node] 6.07s call test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop 6.07s call test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces 5.94s teardown test_mysql57_database_engine/test.py::test_restart_server 5.86s teardown test_placement_info/test.py::test_placement_info_missing_data 5.85s call test_partition/test.py::test_make_clone_in_detached 5.82s teardown test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop 5.81s call test_recovery_replica/test.py::test_recovery 5.61s teardown test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings 5.30s call test_part_uuid/test.py::test_part_uuid 5.13s call test_named_collections/test.py::test_granular_access_create_alter_drop_query 4.82s call test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids 4.76s call test_replicated_merge_tree_encryption_codec/test.py::test_same_keys 4.68s teardown test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout 3.82s call test_quorum_inserts/test.py::test_simple_add_replica 3.57s call test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 3.53s call test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 3.20s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] 3.20s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] 3.13s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] 3.06s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] 3.03s call test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout 3.03s call test_partition/test.py::test_drop_detached_parts 3.02s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] 3.01s call test_replicated_merge_tree_encryption_codec/test.py::test_different_keys 2.98s call test_partition/test.py::test_detached_part_dir_exists 2.97s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] 2.94s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] 2.92s call test_placement_info/test.py::test_placement_info_from_imds 2.82s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] 2.76s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] 2.74s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] 2.66s call test_placement_info/test.py::test_placement_info_from_file 2.65s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] 2.65s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] 2.64s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] 2.59s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] 2.57s call test_partition/test.py::test_attach_check_all_parts 2.56s call test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] 2.54s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] 2.50s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] 2.48s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] 2.48s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] 2.48s call test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] 2.47s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] 2.44s call test_mysql57_database_engine/test.py::test_predefined_connection_configuration 2.38s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] 2.35s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] 2.34s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] 2.33s call test_mysql57_database_engine/test.py::test_mysql_types[float_1] 2.29s call test_mysql57_database_engine/test.py::test_mysql_types[float_2] 2.27s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] 2.24s call test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] 2.23s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] 2.23s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] 2.18s call test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] 2.11s call test_partition/test.py::test_partition_complex 2.11s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] 2.09s call test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] 1.98s call test_named_collections/test.py::test_config_reload 1.88s call test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout 1.64s call test_reload_query_masking_rules/test.py::test_reload_query_masking_rules 1.57s call test_reloading_settings_from_users_xml/test.py::test_force_reload 1.28s setup test_partition/test.py::test_drop_detached_parts 1.16s call test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int 1.14s call test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload 1.06s call test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0] 1.04s setup test_partition/test.py::test_system_detached_parts 1.02s call test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings 0.94s call test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum 0.88s call test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1] 0.87s setup test_partition/test.py::test_partition_complex 0.76s setup test_partition/test.py::test_partition_simple 0.69s call test_placement_info/test.py::test_placement_info_from_config 0.65s call test_partition/test.py::test_cannot_attach_active_part 0.61s call test_mysql57_database_engine/test.py::test_password_leak 0.60s setup test_partition/test.py::test_cannot_attach_active_part 0.55s call test_partition/test.py::test_partition_simple 0.49s call test_placement_info/test.py::test_placement_info_missing_data 0.48s teardown test_partition/test.py::test_drop_detached_parts 0.43s setup test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout 0.38s teardown test_partition/test.py::test_attach_check_all_parts 0.33s setup test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout 0.31s setup test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload 0.29s setup test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int 0.28s setup test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum 0.17s teardown test_partition/test.py::test_partition_simple 0.17s teardown test_partition/test.py::test_partition_complex 0.17s teardown test_partition/test.py::test_cannot_attach_active_part 0.01s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] 0.00s setup test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table 0.00s teardown test_named_collections/test.py::test_granular_access_create_alter_drop_query 0.00s teardown test_named_collections/test.py::test_config_reload 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[float_1] 0.00s teardown test_partition/test.py::test_make_clone_in_detached 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0] 0.00s setup test_named_collections/test.py::test_granular_access_create_alter_drop_query 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] 0.00s setup test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1] 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[float_2] 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] 0.00s teardown test_named_collections/test.py::test_default_access 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[float_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] 0.00s teardown test_recovery_replica/test.py::test_choose_source_replica 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] 0.00s setup test_named_collections/test.py::test_sql_commands[False] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] 0.00s setup test_mysql57_database_engine/test.py::test_password_leak 0.00s teardown test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids 0.00s setup test_quorum_inserts/test.py::test_simple_add_replica 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] 0.00s teardown test_named_collections/test.py::test_show_grants 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[float_2] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] 0.00s setup test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] 0.00s teardown test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] 0.00s setup test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0] 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0] 0.00s setup test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] 0.00s teardown test_named_collections/test.py::test_keeper_storage 0.00s setup test_recovery_replica/test.py::test_recovery 0.00s setup test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] 0.00s teardown test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 0.00s setup test_named_collections/test.py::test_granular_access_show_query 0.00s setup test_named_collections/test.py::test_name_escaping[node_with_keeper] 0.00s setup test_named_collections/test.py::test_name_escaping[node] 0.00s setup test_placement_info/test.py::test_placement_info_missing_data 0.00s setup test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server 0.00s teardown test_reloading_settings_from_users_xml/test.py::test_force_reload 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] 0.00s teardown test_named_collections/test.py::test_sql_commands[False] 0.00s setup test_placement_info/test.py::test_placement_info_from_file 0.00s setup test_replicated_merge_tree_encryption_codec/test.py::test_same_keys 0.00s setup test_named_collections/test.py::test_keeper_storage 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] 0.00s teardown test_named_collections/test.py::test_granular_access_show_query 0.00s teardown test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1] 0.00s setup test_replicated_merge_tree_encryption_codec/test.py::test_different_keys 0.00s setup test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1] 0.00s setup test_recovery_replica/test.py::test_update_metadata 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1] 0.00s teardown test_placement_info/test.py::test_placement_info_from_config 0.00s setup test_partition/test.py::test_detached_part_dir_exists 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] 0.00s setup test_named_collections/test.py::test_sql_commands[True] 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0] 0.00s teardown test_partition/test.py::test_detached_part_dir_exists 0.00s setup test_named_collections/test.py::test_show_grants 0.00s setup test_named_collections/test.py::test_default_access 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1] 0.00s setup test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] 0.00s teardown test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0] 0.00s setup test_mysql57_database_engine/test.py::test_predefined_connection_configuration 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 0.00s setup test_mysql57_database_engine/test.py::test_restart_server 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] 0.00s teardown test_mysql57_database_engine/test.py::test_predefined_connection_configuration 0.00s teardown test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_ttl 0.00s teardown test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] 0.00s setup test_placement_info/test.py::test_placement_info_from_imds 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] 0.00s teardown test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] 0.00s teardown test_mysql57_database_engine/test.py::test_password_leak 0.00s teardown test_recovery_replica/test.py::test_recovery 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_ttl 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] 0.00s setup test_partition/test.py::test_make_clone_in_detached 0.00s teardown test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] 0.00s teardown test_placement_info/test.py::test_placement_info_from_imds 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] 0.00s teardown test_named_collections/test.py::test_name_escaping[node_with_keeper] 0.00s teardown test_named_collections/test.py::test_name_escaping[node] 0.00s teardown test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces 0.00s setup test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 0.00s teardown test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] 0.00s teardown test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] 0.00s teardown test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int 0.00s teardown test_placement_info/test.py::test_placement_info_from_file 0.00s teardown test_replicated_merge_tree_encryption_codec/test.py::test_different_keys =========================== short test summary info ============================ FAILED test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection FAILED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False] FAILED test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True] PASSED test_reloading_settings_from_users_xml/test.py::test_force_reload PASSED test_placement_info/test.py::test_placement_info_from_config PASSED test_partition/test.py::test_attach_check_all_parts PASSED test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout PASSED test_partition/test.py::test_cannot_attach_active_part PASSED test_placement_info/test.py::test_placement_info_from_file PASSED test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum PASSED test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int PASSED test_partition/test.py::test_detached_part_dir_exists PASSED test_placement_info/test.py::test_placement_info_from_imds PASSED test_placement_info/test.py::test_placement_info_missing_data PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids PASSED test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload PASSED test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_keys PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_18] PASSED test_partition/test.py::test_drop_detached_parts PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_19] PASSED test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_1] PASSED test_partition/test.py::test_make_clone_in_detached PASSED test_named_collections/test.py::test_config_reload PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_20] PASSED test_partition/test.py::test_partition_complex PASSED test_replicated_merge_tree_encryption_codec/test.py::test_same_keys PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_21] PASSED test_partition/test.py::test_partition_simple PASSED test_recovery_replica/test.py::test_choose_source_replica PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_22] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_23] PASSED test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_24] PASSED test_recovery_replica/test.py::test_recovery PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_25] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False] PASSED test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_2] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_3] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0] PASSED test_partition/test.py::test_system_detached_parts PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_4] PASSED test_part_uuid/test.py::test_part_uuid PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_5] PASSED test_named_collections/test.py::test_default_access PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_6] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_7] PASSED test_named_collections/test.py::test_granular_access_create_alter_drop_query PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_80] PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_81] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server PASSED test_mysql57_database_engine/test.py::test_mysql_types[common_types_9] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1] PASSED test_reload_query_masking_rules/test.py::test_reload_query_masking_rules PASSED test_recovery_replica/test.py::test_update_metadata PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3] PASSED test_mysql57_database_engine/test.py::test_mysql_types[datetime_default] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1] PASSED test_reload_zookeeper/test.py::test_reload_zookeeper PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_1] PASSED test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1] PASSED test_named_collections/test.py::test_granular_access_show_query PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_2] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_3] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6] PASSED test_mysql57_database_engine/test.py::test_mysql_types[decimal_4] PASSED test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_1] PASSED test_mysql57_database_engine/test.py::test_mysql_types[float_2] PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6] PASSED test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server PASSED test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default] PASSED test_mysql57_database_engine/test.py::test_password_leak PASSED test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper PASSED test_mysql57_database_engine/test.py::test_predefined_connection_configuration PASSED test_mysql57_database_engine/test.py::test_restart_server PASSED test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch PASSED test_named_collections/test.py::test_keeper_storage PASSED test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1] PASSED test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0] PASSED test_named_collections/test.py::test_name_escaping[node] PASSED test_named_collections/test.py::test_name_escaping[node_with_keeper] PASSED test_named_collections/test.py::test_show_grants PASSED test_named_collections/test.py::test_sql_commands[False] PASSED test_named_collections/test.py::test_sql_commands[True] PASSED test_quorum_inserts/test.py::test_insert_quorum_with_ttl PASSED test_quorum_inserts/test.py::test_simple_add_replica =================== 3 failed, 97 passed in 484.56s (0:08:04) =================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_vs3hxi --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=1e0b53d756cf -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_18]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_19]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_20]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_21]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_22]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_23]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_24]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_25]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_5]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_7]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_80]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_81]' 'test_mysql57_database_engine/test.py::test_mysql_types[common_types_9]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_6_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[datetime_default]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_18_6_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_3]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_40_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[decimal_4]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_1]' 'test_mysql57_database_engine/test.py::test_mysql_types[float_2]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_6]' 'test_mysql57_database_engine/test.py::test_mysql_types[timestamp_default]' test_mysql57_database_engine/test.py::test_password_leak test_mysql57_database_engine/test.py::test_predefined_connection_configuration test_mysql57_database_engine/test.py::test_restart_server test_named_collections/test.py::test_config_reload test_named_collections/test.py::test_default_access test_named_collections/test.py::test_granular_access_create_alter_drop_query test_named_collections/test.py::test_granular_access_show_query test_named_collections/test.py::test_keeper_storage 'test_named_collections/test.py::test_keeper_storage_remove_on_cluster[False-expected_raise1]' 'test_named_collections/test.py::test_keeper_storage_remove_on_cluster[True-expected_raise0]' 'test_named_collections/test.py::test_name_escaping[node]' 'test_named_collections/test.py::test_name_escaping[node_with_keeper]' test_named_collections/test.py::test_show_grants 'test_named_collections/test.py::test_sql_commands[False]' 'test_named_collections/test.py::test_sql_commands[True]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-key-1]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_range-sipHash64(key)-1]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-key-1]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-0]' 'test_parallel_replicas_custom_key_load_balancing/test.py::test_parallel_replicas_custom_key_load_balancing[custom_key_sampling-sipHash64(key)-1]' test_part_uuid/test.py::test_part_uuid test_partition/test.py::test_attach_check_all_parts test_partition/test.py::test_cannot_attach_active_part test_partition/test.py::test_detached_part_dir_exists test_partition/test.py::test_drop_detached_parts test_partition/test.py::test_make_clone_in_detached test_partition/test.py::test_partition_complex test_partition/test.py::test_partition_simple test_partition/test.py::test_system_detached_parts test_placement_info/test.py::test_placement_info_from_config test_placement_info/test.py::test_placement_info_from_file test_placement_info/test.py::test_placement_info_from_imds test_placement_info/test.py::test_placement_info_missing_data test_quorum_inserts/test.py::test_drop_replica_and_achieve_quorum 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_drop_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_keeper_loss_connection 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[False]' 'test_quorum_inserts/test.py::test_insert_quorum_with_move_partition[True]' test_quorum_inserts/test.py::test_insert_quorum_with_ttl test_quorum_inserts/test.py::test_simple_add_replica test_recovery_replica/test.py::test_choose_source_replica test_recovery_replica/test.py::test_recovery test_recovery_replica/test.py::test_update_metadata test_reload_max_table_size_to_drop/test.py::test_reload_max_table_size_to_drop test_reload_query_masking_rules/test.py::test_reload_query_masking_rules test_reload_zookeeper/test.py::test_reload_zookeeper test_reloading_settings_from_users_xml/test.py::test_force_reload test_reloading_settings_from_users_xml/test.py::test_reload_on_timeout test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_enum test_reloading_settings_from_users_xml/test.py::test_unexpected_setting_int test_reloading_settings_from_users_xml/test.py::test_unknown_setting_force_reload test_reloading_settings_from_users_xml/test.py::test_unknown_setting_reload_on_timeout test_remove_stale_moving_parts/test.py::test_remove_stale_moving_parts_without_zookeeper test_replicated_fetches_bandwidth/test.py::test_limited_fetch_single_table test_replicated_fetches_bandwidth/test.py::test_limited_fetches_for_server test_replicated_fetches_bandwidth/test.py::test_limited_send_single_table test_replicated_fetches_bandwidth/test.py::test_limited_sends_for_server test_replicated_fetches_bandwidth/test.py::test_should_execute_fetch test_replicated_merge_tree_config/test.py::test_replicated_merge_tree_settings test_replicated_merge_tree_encryption_codec/test.py::test_different_current_key_ids test_replicated_merge_tree_encryption_codec/test.py::test_different_keys test_replicated_merge_tree_encryption_codec/test.py::test_different_nonces test_replicated_merge_tree_encryption_codec/test.py::test_same_keys -vvv" altinityinfra/integration-tests-runner:37a9815fd2fa ' returned non-zero exit status 1.